NIST Language Technology Evaluation Cookbook
نویسندگان
چکیده
We review some of the methodology applied to the various NIST language technology evaluations. We discuss the elements included in each evaluation plan, and suggest what we believe are key practices for successful evaluations, and what pitfalls should be avoided. A couple of lessons learned are noted.
منابع مشابه
NIST language recognition evaluation - plans for 2015
We discuss two NIST coordinated evaluations of automatic language recognition technology planned for calendar year 2015 along with possible additional plans for the future. The first is the Language Recognition i-Vector Machine Learning Challenge, largely modeled on the 2013-2014 Speaker Recognition i-Vector Machine Learning Challenge. This online challenge, emphasizing the language identificat...
متن کاملNIST Rich Transcription 2002 Evaluation: A Preview
The National Institute of Standards and Technology (NIST) has been implementing evaluations of automatic speech transcription technologies for over 15 years. NIST has helped guide progress in these technologies by: creating increasingly challenging and realistic tests, helping to provide associated linguistic resources, employing uniform metrics and analyses across systems to assess performance...
متن کاملNIST 2007 Language Recognition Evaluation: From the Perspective of IIR
This paper describes the Institute for Infocomm Research (IIR) system for the 2007 Language Recognition Evaluation (LRE) conducted by the National Institute of Standards and Technology (NIST). The submitted system is a fusion of multiple state-ofthe-art language classifiers using diversified discriminative language cues. We implemented several state-of-the-art algorithms using both phonotactic ...
متن کاملJAVA-Based XML Utility for the NIST Machine Tool Data Repository
The National Institute of Standards and Technology (NIST) is developing a specification for a standard format to represent machine tool performance test data. This data is used for machine acceptance, predictive maintenance, error compensation, and to evaluate the capability of a machine or contractor to manufacture parts to specified tolerances. This paper describes the implementation of a JAV...
متن کاملA human benchmark for the NIST language recognition evaluation 2005
In this paper we describe a human benchmark experiment for language recognition. We used the same task, data and evaluation measure as in the NIST Language Recognition Evaluation (LRE) 2005. For the primary condition of interest all 10-second trials were used in the experiment. The experiment was conducted by 38 subjects, who each processed part of the trials. For the seven-language closed set ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004